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Additionally, this concept is applied to the encoding of 
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VECTOR WAVELET ORGANIZATION 

This application claims the benefit of U.S. Provisional 
Application No. 60/007,012 filed Oct. 25, 1995. 

This application is a divisional of application Ser. No. 5 
08/736,114, filed Oct. 24, 1996, currently pending, which 
claims the benefit of the filing date of provisional application 
No. 60/020,852, filed Jun. 28, 1996. 

The invention relates to image processing systems and, 
more particularly, to image processing systems that utilize 
wavelet-based coding techniques. 

BACKGROUND OF THE DISCLOSURE 

Data compression systems are useful for representing 
information as accurately as possible with a rmnimum 
number of bits and thus minimizing the amount of data 
which must be stored or transmitted in an information 
storage or transmission system. One of the primary means of 
compression used in the art removes redundant information 
from the original data without significantly impacting the 
quality of the decompressed data when compared to the 
original data. 

One such compression technique appears in the Proceed- 
ings of the International Conference on Acoustics, Speech 
and Signal Processing, San Francisco, Calif. March 1992, 
volume IV, pages 657-660, where there is disclosed a signal 
compression system which applies a hierarchical subband 
decomposition, or wavelet transform, followed by the hier- 
archical successive approximation entropy-coded quantizer 
incorporating zerotrees. The representation of signal data 
using a multiresolution hierarchical subband representation 
was disclosed by Burt et al. in IEEE Trans, on Commun., Vol 
Com-31, No. 4, April 1983, page 533. A wavelet pyramid, 
also known as critically sampled quadrature-mirror filter 
(QMF) subband representation, is a specific type of multi- 
resolution hierarchical subband representation of an image. 
A wavelet pyramid was disclosed by Pentland et al. in Proc. 
Data Compression Conference Apr. 8-11, 1991, Snowbird, 
Utah. A QMF subband pyramid has been described in 
"Subband Image Coding", J. W. Woods ed., Kluwer Aca- 
demic Publishers, 1991 and I. Daubechies, Ten Lectures on 
Wavelets, Society for Industrial and Applied Mathematics 
(SIAM): Philadelphia, Pa, 1992. 

Wavelet transforms, otherwise known as hierarchical sub- 
band decomposition, have recently been used for low bit rate 
image compression because such decomposition leads to a 
hierarchical multi-scale representation of the source image. 
Wavelet transforms are applied to an important aspect of low 
bit rate image coding: the coding of a binary map (a wavelet 
tree) indicating the locations of the non-zero values, other- 
wise known as the significance map of the transform coef- 
ficients. Using scalar quantization followed by entropy 
coding, in order to achieve very low bit rates, i.e., less than 
1 bit/pel, the probability of the most likely symbol after 
quantization — the zero symbol — must be extremely high. 
Typically, a large fraction of the bit budget must be spent on 
encoding the significance map. It follows that a significant 
improvement in encoding the significance map translates 
into a significant improvement in the compression of infor- 
mation preparatory to storage or transmission. 

To accomplish this task, a new structure called a zerotree 
has been developed. A wavelet coefficient is said to be 
insignificant with respect to a given threshold T, if the 
coefficient has a magnitude less than or equal to T. The 
zerotree is based on the hypothesis that if a wavelet coef- 
ficient at a coarse scale is insignificant with respect to a 
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given threshold T, then all wavelet coefficients of the same 
orientation in the same spatial location at finer scales are 
likely to be insignificant with respect to T. Empirical evi- 
dence suggests that this hypothesis is often true. 

More specifically, in a hierarchical subband system, with 
the exception of the highest frequency subbands, every 
coefficient at a given scale can be related to a set of 
coefficients at the next finer scale of similar orientation 
according to a structure called a wavelet tree. The coeffi- 
cients at the coarsest scale will be called the parent nodes, 
and all coefficients corresponding to the same spatial or 
temporal location at the next finer scale of similar orienta- 
tion will be called child nodes. For a given parent node, the 
set of all coefficients at all finer scales of similar orientation 
corresponding to the same location are called descendants. 
Similarly, for a given child node, the set of coefficients at all 
coarser scales of similar orientation corresponding to the 
same location are called ancestors. With the exception of the 
lowest frequency subband, all parent nodes have four child 
nodes. For the lowest frequency subband, the parent-child 
relationship is defined such that each parent node has three 
child nodes. 

Nodes are scanned in the order of the scales of the 
decomposition, from coarsest level to finest. This means that 
no child node is scanned until after its parent and all other 
parents in all subbands at the same scale as that parent have 
been scanned. This is a type of modified breadth-first, 
subband by subband, traversal performed across all the 
wavelet trees defined by the coefficients of the wavelet 
transform of the two-dimensional data set. 

Given a threshold level to determine whether or not a 
coefficient is significant, a node is said to be a ZEROTREE 
ROOT if 1) the coefficient at a node has an insignificant 
magnitude, 2) the node is not the descendant of a root, i.e., 
it is not completely predictable from a coarser scale, and 3) 
all of its descendants are insignificant. A ZEROTREE 
ROOT is encoded with a special symbol indicating that the 
insignificance of the coefficients at finer scales is completely 
predictable. To efficiently encode the binary significance 
map, four symbols are entropy coded: ZEROTREE ROOT, 
ISOLATED ZERO, and two non-zero symbols, POSITIVE 
SIGNIFICANT and NEGATIVE SIGNIFICANT. 

U.S. Pat. No. 5,412,741 issued May 2, 1995 and herein 
incorporated by reference discloses an apparatus and method 
for encoding information with a high degree of compression. 
The apparatus uses zerotree coding of wavelet coefficients in 
a much more efficient manner than any previous techniques. 
The key to this apparatus is the dynamic generation of the 
list of coefficient indices to be scanned, whereby the 
dynamically generated list only contains coefficient indices 
for which a symbol must be encoded. This is a dramatic 
improvement over the prior art in which a static list of 
coefficient indices is used and each coefficient must be 
individually checked to see whether a) a symbol must be 
encoded, or b) it is completely predictable. 

The apparatus disclosed in the '741 patent uses a method 
for encoding information comprising the steps of forming a 
wavelet transform of the image, forming a zerotree map of 
the wavelet coefficients, encoding the significant coefficients 
on an initial dominant list from the coarsest level of the 
transform and the children of those coefficients whose 
indices are appended to the dominant list as the coefficient 
of the parent is found to be significant, reducing the 
threshold, refining the estimate of the value of the significant 
coefficients to increase the accuracy of the coded 
coefficients, and cycling back to scan the dominant list anew 
at the new, reduced threshold. 



08/17/2004, EAST Version: 1.4.1 



US 6,298,167 Bl 

3 4 

To accomplish the iterative process, the method of the FIG. 15 depicts an object parent-child relationship; and 

'741 patent is accomplished by scanning the wavelet tree F IG. 16 depicts the reorganization of the coefficient 

subband by subband, i.e., all parent nodes are coded, then all vectors of each wavelet tree to form a "wavelet block" 

children, then all grandchildren and so on and they are To understandi identical reference numerals 

encoded bit-plane by bit-plane. As the process iterates 5 hftve been ^ wh(Jre & ssib , desi identica] 

through the wavelet tree representation of the image, this elements , hat afe common |J , 0 the fi 
apparatus codes one of four symbols within the zerotree 

map. Any improvement in the speed at which a wavelet tree DETAILED DESCRIPTION 

is processed would be advantageous. . , . . , , ,. * , r , 
* - , . . _ . tin FIG. 1 depicts a block diagram of an encoder 100 of the 

Therefore there is a need in the art for an improved 10 { inyenlion ^ FI(J 2 d kls a flowchaft ^ 

method of classifying and coding the nodes of a wavelet tree tatiofl of the atioQ of the eQCoder 10Q of mQ L Tq ^ 

that leads to more efficient coding and rapid processing. uoders tand the invention, the reader should simultaneously 

SUMMARY OF THE INVENTION consult both FIGS. 1 and 2 while reading the following 

description of the invention. 

The present invention is apparatus and a concomitant The encoder 100 contains a wavelet tree generator 104, an 

method of encoding zerotrees in a wavelet-based coding optional wavelet tree reorg anizer m , a quantizer 110, a 

technique. Specifically, the invention uses a depth-first pat- symbol assignor 112 , and ^ en tropy encoder 114. Each of 

tern for traversing the zerotree, i.e., each branch of the tree, these components is connected in series to process an image 

from parent to child to grandchild and so on, is fully at port 102 into a coded output image at port U 6. The input 

traversed before a next branch is traversed. The depth-first ^ typically a pixelated (digitized) photographic image 

tree traversal pattern is used to quantize the coefficients of as can be prod uced from an image scanner or a computer 

the tree as well as to assign symbols to the quantized graphics system. However, the input image can also be a 

coefficients. Additionally, the invention assigns one of three frame a series of frames of video images or a motion 

symbols to each node: ZEROTREE ROOT, VALUED compensated residuals frame produced by a video encoding 

ZEROTREE ROOT, and VALUE. By using three symbols system> In generalj me invention proC esses any form of 

and the efficient tree traversal pattern, the invention is digitized image or portion thereof. Thus, the method of 

substantially more efficient at encoding a zerotree than the operation generally begins at step 202 with the input of an 

prior art. Additionally, this concept is also applied to the "image", i.e., any form of two-dimensional data, 
encoding of "vector" zerotrees. 3o ^ wave[et ^ generatof 1Q4 performs (&t step m & 

BRIEF DESCRIPTION OF THE DRAWINGS wavelet hierarchical subband decomposition to produce a 

conventional wavelet tree representation of the input image. 

The teachings of the present invention can be readily To accomplish such image decomposition, the image is 

understood by considering the following detailed descrip- decomposed using times two subsampling in each of two- 
tion in conjunction with the accompanying drawings, in 35 dimensions into high horizontal-high vertical (HH), high 

which : horizontal-low vertical (HL), low horizontal-high vertical 

FIG. 1 is a block diagram of an image encoder of the (LH), and low horizontal-low vertical (LL), frequency sub- 
present invention; bands. The LL subband is then further subsampled times two 

FIG. 2 is a flowchart illustrating the encoding method of m each of ^ dimensions to produce a set of HH, HL, LH 
the encoder shown in FIG. 1; 40 and LL subbands. This subsampling is accomplished recur- 

FIG. 3 is a schematic illustration of parent-child depen- sivel y t0 P roduce an arra y of subbands such as that illus- 

dencies of subbands in an image decomposed to three scales tratcd m FIG ' 3 wbcrc thrce ^samplings have been used, 

within a wavelet tree* Preferably four or more subsamp lings are used in practice. 

a a * + *u * u-u I*- u- r .i_ The parent-child dependencies between subbands are illus- 

Jl^L^^r^^ 'elauonship ** three 45 tfated as ^ ^ from ^ subband 

generations of a subsampled image; Qodes to the subbands of the child nodes ^ ^ 

FIG. 5 depicts a schematic representation of the interre- frequency subband is the top left LU and the highest 

lation of various nodes within a wavelet tree; frequency subband is at the bottom right HHj. In this 

FIG. 6 depicts a wavelet block representation of a wavelet example, all child nodes have one parent. A detailed dis- 

tree i 50 cussion of subband decomposition is presented in J. M. 

FIG. 7 depicts a flowchart of a quantization routine Shapiro, "Embedded Image Coding Using Zerotrees of 

executed by the present invention; Wavelet Coefficients", IEEE Trans, on Signal Processing, 

FIG. 8 depicts a flowchart of a symbol assignment routine Vo1 - 41 > No * 12 > PP- 3445-62, December 1993. 

executed by the present invention; FIG. 4 depicts the parent -child relationship for three 

FIG. 9 depicts a block diagram of one embodiment of a 55 generations of a subsampled image. A single parent node 

vector-wavelet transform; 400 has four child nodes 402 corresponding to the same 

FIG. 10 depicts a block diagram of another embodiment region 1x5 the ijna S e with times four subsampling, i.e., times 

of a vector-wavelet transform- ^ subsampling in each of two dimensions. Each child 

FIG. 11 depicts a block diagram ofone embodiment of a D °* e ^ ^ four corresponding next generation child 

scalar-wavelet transform followed by vectors or object 60 D ° deS T W1 * 3 further * meS ^ ^samphng. The 

&QU ^ 7 J relationship, or data structure, that relates a parent node to its 

vin A A . . . . , r . . children and grandchildren is a wavelet tree. Note that each 

FIG. 12 depicts a block diagram of another embodiment , or M in lhe low . low subband Qas a associated 

of a^calar-wavelet transform followed by vectors or object ^ it Howevefj the plurality of tfees that extend from ^ 

gr ^ Ln ^ 65 low-low subband taken together are generally discussed in 

FIG. 13 depicts an example of a vector wavelet; me art as "the" wavelet tree for the image. This disclosure 

FIG. 14 depicts a vector parent-child relationship; will also follow this nomenclature. 
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As shown in FIGS. 1 and 2, the quantizer 110 quantizes are not physically rearranged, but are rather remapped into 

(at step 210) the coefficients of the wavelet tree in a a virtual memory. Thus, an index into the physical memory 

"depth-first" pattern. A depth-first pattern is very dissimilar is created, where the index (virtual memory) has memory 

from the prior art encoding systems in that the prior art locations that are arranged into wavelet blocks. For each 

encoding systems traverse the wavelet tree in a modified 5 access { n { 0 the index, the address into the index is mapped 

breadth-first pattern across all subbands, i.e., quantize all l0 a physical memory location where the coefficient is 

parent nodes, then all children, then all grandchildren and so stored ThuSj by a vj rtua i memory approach, the advantages 

on. In contrast, the inventive method traverses each tree in of wavelet bk)cks are availabk withoul physicaUy rearrang . 

"depth-first order i.e., from the root in the low-low sub- ^ the mtmxdetsSs ^ memory . 

band (LL 3 ) through the children. FIG. 5 depicts the depth- ° , „ . 

first pattern used to traverse each tree. For example, begin- 10 L1 B ? ^ a depth-first scanning pattern, each wavelet 

ning at node 500 in LL3 and following the bold path, the 1S U ^mpletely scanned to quantize its coefficients 

inventive depth-first process proceeds to node 502 in sub- . next bi ° C \ 15 scann ? ! u and u f °^*° T 

band LH 3 and then to node 504 in subband LH 2 . From node * °, ck 6 ]° 18 completely scanned, then block 612 then block 

504, the depth-first traversal process successively continues „ 614 > * nd u f ? n ™ a ^ ^ E?? th f° Ugh ^ fram \° f 

to nodes 506, 508, 510 and 512 within subband LH,, i.e., all 15 wavelet blocks ' ^ orderm S of blocks does not have to be 

the children of node 504, then continues on to the siblings m a rast f r scan P attern > but be an y order 88 desired b ? 

of 504 (514, 524, 534) where the four children of each J be W^ation. This includes object-oriented whereby 

sibling are traversed before the next sibling and its children. blocfe corresponding to certain objects are scanned and 

Once this entire branch of the tree is traversed, the traversal , n coded before other objects Since an entire block is located 

process proceeds to another child node of node 500, for 20 at ^secutive memory addresses, the block can easily be 

example, node 544. From that node, the depth-first traversal scanned m a to P down or bottoms U P P attem by selecting 

process proceeds to nodes 546, 548, 550, 552 and 554 before either ! he fi f °[ ^t memory entry for a given block and 

going on to node 556 and so on. accessing all other addresses in ascending or descending 

As each branch is traversed, the coefficients are quantized 2 s 

into discrete values. Any quantization approach can be used Importantly, with such reorganization, each wavelet block 

with the present invention. The quantization process maps a caD be assigned a different quantizer scale based on its 

continuous coefficient value to a discrete value having either s P atial location m tne frame - This permits the quantizer 110 

a positive value, a negative value or zero value. In sum, in 10 be allocated specifically for a spatial location of the 

a depth-first scan pattern, children 506, 508, 510, and 512 30 coefficients and/or in accordance with the frequency band 

are scanned after their parent 504 and before any of the represented by the coefficient. As such, the scale of the 

neighboring parents 514, 524 and 534. In this manner, all quantizer can be different across an image such that the 

coefficients that represent a given spatial location are ceoter of the or certain objects within the image can 

scanned, in ascending frequency order from parent 500 to be more accurately quantized than the edges. Similarly, the 

child 502 to grandchild 504 and so on, before the coefficients 35 quantizer scale could be frequency dependent such that 

of the next adjacent spatial location is scanned. ni S her frequency (or, for that matter, lower frequencies, 

Although the foregoing description of the depth-first middle frequencies, various frequency bands, and the like) 

scanning pattern was discussed as a "top down" pattern, a can be <l uantlzed usin g a scale that ls different from other 

depth-first scanning pattern also includes scanning from the frequencies. 

bottom up. As such, the quantization can also be accom- 4D In a particular application, such as a video-phone, the 

phshed by starting at a tree's "leaves" (the bottom-most foreground object(s), e.g., the head and shoulders of a caller, 

nodes) and proceeding up the tree. Using the example of are elected for quantization with more accuracy than the 

FIG. 5, in a "bottom up" pattern, nodes 506, 508, 510 and background objects. Consequently, the important informa- 

512 would be quantized first, then node 504, and so on up uon ^ accurately coded and transmitted, while background 

the tree to 500 last. Once that tree was complete, the 45 information is less accurately coded. The use of wavelet 

quantization process would quantize another tree, and blocks allows for easy bifurcated coding of foreground and 

another, and so on until all the nodes in all the trees were background information. Once a region is selected for 

quantized. As shall be discussed below, the invention oper- accurate coding, the invention can flag the blocks that 

ates more efficiently when using a bottoms up pattern than require accurate coding. As such, the quantizer quantizes 

the top down. 50 mosc blocks using a finer quantization step than is used on 

To facilitate this depth-first scanning pattern, the inven- the unflagged blocks, 

tion reorganizes the quantized coefficients of each wavelet Although wavelet blocks form an intuitive data structure 

tree to form a "wavelet block". As shown in FIGS. 1 and 2, for implementing the invention, use of wavelet blocks is not 

the reorganization is accomplished (at step 206) in the necessary to implementation of the inventive encoder 100. 

wavelet tree reorganizer 108 prior to quantization. 55 As shall be discussed below, the conventional tree structure 

FIG. 6 schematically depicts a wavelet block 604 that is can be used in conjunction with the improved tree traversal 

generated by the invention. The invention maps a tree 602 process and the improved coding technique of the present 

extending from a pixel 600 in the low-low band 606 (LL3) invention. As such, FIGS. 1 and 2 depict the optional nature 

in the wavelet tree 602 into a wavelet block 604. Each of the reorganizer as path 106 and path 208 which respec- 

wavelet block 604 of an image frame 608 comprises those 60 tive ly bypass the reorganizer and its associated function, 

coefficients at all scales and orientations that represent the After quantization, at each node of the tree, the quantized 

frame at the spatial location of the block within the frame. coefficient has either a zero value or a non-zero value. 

The reorganization is accomplished by physically remap- "Zerotrees" exist wherever the coefficient at a node is zero 

ping the memory locations of the coefficients to new and all its descendants form zerotrees, i.e., all descendant 

memory locations that form the wavelet blocks. As such, all 65 nodes have a zero value. The quantized coefficients of the 

the coefficients of a given wavelet block are stored at wavelet tree are efficiently encoded by again scanning each 

sequential address locations. Alternatively, the coefficients tree in a depth-first manner. Thus, symbol assignor 112 



08/17/2004, EAST Version: 1.4.1 



US 6,298,167 Bl 

7 8 

operates (at step 212) by traversing the tree and assigning selects, at step 716, a new (next) tree or quantization. If the 

particular symbols to each node depending upon the node's query at step 714 is affirmatively answered, the routine 

quantized value as well as the quantized values of each proceeds to step 718. At this point in routine 700, all the 

node's descendants. nodes in all the trees have been quantized and assigned a 

Specifically, at each node, the inventive method assigns 5 preliminary symbol. At step 718, the routine 700 calls 

one of three symbols: ZEROTREE ROOT, VALUED symbol assignment routine 800 of FIG. 8. After the symbols 

ZEROTREE ROOT, and VALUE. A ZEROTREE ROOT are assigned, routine 700 ends at block 720. 

denotes a coefficient that is the root of a zerotree. After the n OAn 4t . . . ■ * . 

.... U1 -j.u *j * Routine 800 scans the trees in a top down pattern, i.e., root 

scan in which symbols are assigned, the zerotree does not 4 . IT 4 . * K A 

n . . . M ^ , „ , , ? ' . . . „ to leaves. However, the routine does not need to access every 

need to be scanned any further because it is known that all in , , ; « « c 3 

coefficients in the tree have the value zero A VALUED node because thc trees are P mned at cach occurrence of a 

ZERO^E ROOT t a n4 ^ \TL^IT, f™™* R ° 0T 5 V f V ™ f R °°? 

non-zero value and all four children are ZEROTREE ff^S* 7 T ™ T ? . ste f 802 and P roc , e «> s 

ROOTS. The coding scan of this tree never progresses ,0 s ' eP - " f . ' f r0U T* 4 T 

below this node. A VALUE symbol identifies a coefficient „ «*? Mt ^ m l * e tree ° f ^ Dtized coefficients .At step 

with a value, either zero or non-zero, but also with some " 806 ' ^ routine retrieves the prelumnary symbol in the mark 

descendant somewhere further along the tree that has a map that corresp^ds to the retrieved coeffiaent. The rouhne 

I & queries at step 808 whether the preliminary symbol is a 

non-zero vaiue. potential ZEROTREE ROOT. If the query is affirmatively 

To most efficiently scan the trees to quantize and assign aasweredj the routine at st lhe ZEROTREE 

symbols to the nodes, the quantizer operates in conjunction 20 R0QT bol tQ ^ node ^ * ^ ^ routine 

with the symbol assignor. Fia 7 depicts a detailed flowchart ^ . ^ routine ; ^ flodes Mqw ^ 

of a quantization routine 700 used to quantize the coeffi- ZEROTREE ROOT node because, by definition, all the 

cients of a zerotree and FIG. 8 depicts a detailed flowchart nodes haye a zem valuc 

of a symbol assignment routine 800 for assigning symbol ^ Mrt 

values to represent the quantized coefficient values. 25 ^ routine ^ enes at ste P 820 whether aU Dodes have 
The routine 700 begins at block 702 and proceeds to step beeD Se l eCt u ed ' K . the ^ at , Ste P * negatively 
704 where a coefficient value is retrieved from a node in a ™ red ' *e routine proceeds along the NO path to step 
wavelet tree. As shall be discussed below, the quantization 814 ' * fP * 14 ' 106 ^Uneselects the next node after any 
routine scans the wavelet tree in a bottom up, depth-first P^ ed branches are skipped, in the tree such that a top 
pattern. Thus, the first address is always in the highest 30 down » de P th ^st scan is accomplished, 
frequency subband and, with each iteration through the If the query at step 808 is negatively answered, the routine 
routine, the routine proceeds up the tree to lower and lower proceeds along the NO path to step 816. At step 816, the 
frequency subbands. As the quantized values are generated, routine q ueries whether the mark map contains a potential 
the routine keeps track of the quantized values of the child s y mbo1 of Potential VALUED ZEROTREE ROOT. If the 
nodes, i.e., are the children valued or zero. At step 706, the 35 °* uer y at ste P 816 B affirmatively answered, the routine, at 
routine quantizes the retrieved coefficient value into a posi- ste P 822 > a VALUED ZEROTREE ROOT symbol to 
tive value, a negative value, or zero value. At step 708, a the node > P uts the value on a ^ of non-zero values, and 
mark map is updated with a preliminary symbol value for the P™^ the tree at ste P 824 - ^ routine °i ueries at ste P 820 
node associated with the coefficient value just quantized. whether all nodes have been selected. If the query at step 820 
The mark map symbol depends upon the value of the child 40 ta ne g alivel y answered, the routine proceeds to step 814. 
nodes as well as the value of the present node. Note that, ±c routine, at step 814, selects the next node for 
because the scan is accomplished bottom up, the mark map symbol assignment, skipping the pruned branches, 
is not capable of conclusively indicating whether a node is If the query at step 816 is negatively answered, the routine 
a ZEROTREE ROOT or not. Consequently, after all the assigns, at step 818, a VALUE symbol to the node, and puts 
nodes are assigned a preliminary symbol value, the tree is 45 a value on the list of values that include the value zero. The 
scanned again in a top down pattern to conclusively assign routine queries at step 820 whether all nodes have been 
symbol values. The mark map is an index of the wavelet tree selected. If the query at step 820 is negatively answered, the 
nodes which is filled by the quantization routine 700. At routine proceeds to step 814. Then, the routine selects, at 
each address in the mark map, the routine stores a prelimi- ste P 814, the next node for symbol assignment, 
nary symbol: potential VALUE, potential VALUED 50 The assignment routine continues until all nodes have had 
ZEROTREE ROOT, or potential ZEROTREE ROOT. If the symbols assigned to them. Thus, if the query at step 820 is 
quantized coefficient value has a value, the mark map affirmatively answered, the routine proceeds to step 826, 
location for that coefficient is marked with a potential where the routine 800 ends or returns to routine 700. 
VALUE symbol. If the quantized coefficient value is zero The symbols and values are encoded (at step 214) using 
value and all of that nodes children are zero valued, then the 55 an entropy coder 114, such as a conventional arithmetic 
mark map location is marked with a potential ZEROTREE coder. One possible way to accomplish encoding is as 
ROOT Lastly, if the quantized value has a value and its follows. The symbols are encoded using a three-symbol 
children are all zero valued, then the mark map location is alphabet. The list of non-zero values that correspond one- 
marked with a potential VALUED ZEROTREE ROOT. to-one to the VALUED ZEROTREE ROOT symbols is 
At step 710, the routine queries whether all the nodes in 60 encoded using an alphabet that does not include the value 
the tree have been quantized. If the query is negatively zero. The remaining coefficients, which correspond one-to- 
answered, the routine proceeds to step 712 where a new one to the VALUE symbols, are encoded using an alphabet 
(next) node in the tree is selected for quantization. The that does include the value zero. For any node reached in a 
routine then returns to step 704. If the query at step 710 is scan that is a leaf with no children, neither root symbol could 
affirmatively answered the routine proceeds to step 714. The 65 apply. Therefore, some bits can be saved by not encoding 
routine queries at step 714 whether all the trees have been any symbol for this node and encoding the coefficient using 
quantized. If the query is negatively answered, the routine the alphabet that includes the value zero. An illustrative 
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encoder using a three-symbol coding alphabet for the sym- 
bols and a multi-symbol alphabet for the values would 
follow that disclosed in Witten et al., "Arithmetic Coding for 
Data Compression", Comm. of the ACM, Vol. 30, No. 6, pp. 
520-540, June 1987. In fact, those skilled in the art will 5 
realize that the present invention can be modified by simply 
encoding only the values (or representations of those values) 
of the coefficients in accordance with the assigned symbols. 
Namely, only the values of the coefficients are encoded 
without having to encode the symbols that indicated the 
importance of the coefficients. 

The encoder 100 generates (at step 216) the coded output 
image at port 116. Through utilization of the present 
invention, an image is rapidly and efficiently coded using 
one of three symbols at each node of a wavelet tree plus bits 5 
to encode coefficient values. 

A plurality of alternate embodiments of the present inven- 
tion applies the present coding scheme to vector or object 
zerotree coding for images or video. Namely, the present 
coding scheme, using a wavelet transform, is applied to the 2 q 
concept of vector (or object) zerotree coding which entails 
quantization and zerotree entropy coding of the quantized 
wavelet coefficients. Each of the vector or object zerotree 
coding embodiments of the present invention constitutes a 
vector or object approach to using zerotrees to encode 25 
wavelet coefficients for images or video residuals. Associ- 
ated with each node of each wavelet tree is a list of 
coefficients (a vector) rather than a single coefficient. Tree 
nodes are scanned, coefficients quantized, and symbols 
entropy coded all using a method called Vector ZeroTree 30 
Entropy coding. 

The various vector or object zerotree coding embodiments 
fall into two categories, scalar-wavelet transformation fol- 
lowed by vectors (or objects) grouping, or vectors (or 
objects) grouping followed by vector-wavelet transforma- 35 
tion. Namely, an image, a video frame, or a frame of 
motion-compensated video residuals is first either scalar- 
wavelet transformed or vector- wavelet transformed. FIGS. 
11-12 illustrate the first category where the scalar wavelet is 
first computed, then wavelet coefficients are grouped into 40 
vectors or objects after the transformation. In contrast, FIGS. 
9-10 illustrate the second category, where the inputs are 
grouped as vectors and are passed through a vector-wavelet 
transform, to produce the outputs as vectors. In both cases, 
the result can be viewed as a frame of vectors or objects of 45 
wavelet coefficients. 

Briefly, in two embodiments of the invention (FIGS. 9 and 
10), vectors are defined by a fixed tiling of the 2-D matrix 
of image or video pixels or residuals and the wavelet 
transform computed is a vector wavelet transform. Then, 50 
either vector quantization or scalar quantization is combined 
with vector zerotree entropy coding to yield a coded output 
image. 

In another four embodiments of the invention (FIGS. 11 
and 12), a scalar wavelet transform is computed and then 55 
either vectors are defined by a fixed tiling of the 2-D matrix 
of wavelet coefficients at the coarsest scale or objects of 
arbitrary size and shape are determined by a segmentation 
algorithm (i.e., producing two embodiments). In turn, the 
vectors or objects are either vector quantized or scalar 
quantized in combination with vector zerotree entropy cod- 
ing to yield a coded output image (i.e., producing another 
two embodiments for a total of four embodiments). One 
form of a vector wavelet is disclosed by W. Li and Y.-Q. 
Zhang, "Vector-Based Signal Processing and Quantization 
for Image and Video Compression", Proceedings of the 
IEEE, vol. 83, pp. 317-335, February 1995. 
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FIGS. 13 illustrates an example of a vector wavelet. A 
vector wavelet of an image or video residual is a transform 
whereby adjacent samples of the image or video residual are 
grouped together to form vectors and then the wavelet 
transform is applied to those vectors. A vector wavelet 
transform is computed by the following steps: subsampling 
the frame to obtain a number of subsampled frames of 
spatially offset frame-representative signals; transforming 
each of the subsampled frames by a discrete wavelet trans- 
form to obtain a corresponding number of frames of wavelet 
transform coefficient-representative signals; and forming 
vector-representative signals from corresponding 
coefficient-representative signals of the respective frames of 
coefficient-representative signals. 

Namely, the frame (input image) 1302 is decomposed into 
its 2-D polyphase components. Each polyphase component 
is a spatially-offset frame of the original frame as illustrated 
by the four (4) delta functions 1304.t-1304 4 . Each spatially- 
offset frame is then subsampled with times four 
subsampling, i.e., times two subsampling in each of two 
dimensions by subsampler 1306 to produce the polyphase 
component. In turn, a wavelet transform 1308 is applied to 
each polyphase component, and then vectors are formed by 
grouping together the transform coefficients, one from each 
wavelet-transformed component, with the same indices. In 
this manner, vectors of mxn pixels are transformed into 
vectors of mxn coefficients 1310. 

Vector wavelet transform can also be viewed as a number 
of overlaying frames and each vector or object comprises 
one coefficient from each frame where those coefficients all 
share the same index. The number of frames is equal to the 
dimension of the vectors or size of the objects. 

The various embodiments are now discussed in detail 
with reference to FIGS. 9-12. More specifically, FIGS. 
11-12 illustrate four embodiments that apply scalar- wave let 
transformation followed by vectors (or objects) grouping. 
Since the encoders of FIGS. 11 and 12 share many 
similarities, the various embodiments are discussed simul- 
taneously below through the use of numerical labels. 

The encoder 1100 (1200) contains a wavelet tree genera- 
tor 1103 (1203), a vectors or objects organizer 1104 (1204) 
, an optional vector/object wavelet tree reorganizer 1108 
(1208), a vector quantizer 1110 (a scalar quantizer 1210 with 
a post-quantization processor 1211), a symbol assignor 1112 
(1212), and an entropy encoder 1114 (a vector entropy 
encoder 1214). Each of these components is connected in 
series to process an image at port 1102 (1202) into a coded 
output image at port 1116 (1216). 

The wavelet tree generator 1103 (1203) performs a wave- 
let hierarchical subband decomposition to produce a con- 
ventional wavelet tree representation of the input image 
1102 (1202) as discussed above. In turn, vectors or objects 
organizer 1104 (1204) defines vectors by a fixed grouping of 
adjacent coefficients in the low-low band of the wavelet 
transform. Objects are defined by applying a segmentation 
algorithm to the wavelet coefficients of the low-low band. 
The segmentation is assumed to be complete and nonover- 
lapping. Therefore, each coefficient in the low-low band is 
an element in one and only one vector or object. Various 
segmentation algorithms can be applied to the present 
invention, such as disclosed by Patrick Campbell McLean, 
in "Structured Video Coding", master thesis submitted to the 
Media Arts and Science Section, Massachusetts Institute of 
Technology, June 1991 or pending U.S. patent application 
Ser. No. 08/339,491 filed on Nov. 14, 1994. 

Referring to FIGS. 14-15, the vector 1402 or object 1502 
is propagated into higher bands by upsampling to define an 
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outline 1404 (1504) at the next finer scale. In order to node of each tree, a symbol assignor 1112 (1212) assigns a 

maintain the vectors and/or objects at the same size as in the symbol to characterize the quantized vector at that node. The 

coarser scale, the coefficients of each vector or object within symbols and the quantized coefficient values are all encoded 

the outline are regrouped into four vectors 1406 or objects using an entropy coder 1114 (1214), such as an arithmetic 

1506 as illustrated in FIGS. 14-15. The vector or object at 5 coder. 

the coarser scale is called the vector or object parent and the After either type of quantization, zerotrees exist wherever 

four vectors or objects at the next finer scale are called t he coefficient vector at a node is the zero vector and all 

vector or object children. The four vector/object children are descendants are zerotrees. The wavelet trees are efficiently 

extracted from the outline by taking the four polyphase represented and coded by assigning symbols to represent the 

components 1408 (1508). This is equivalent to a shifted id no des and by pruning the trees at zerotrees. An arithmetic 

subsampiing of the outline. Examples of the vector parent- coc j cr j s use d t0 assign the minimum number of bits to only 

child relationship and object parent-child relationship are those symbols and values needed by the decoder to faithfully 

shown in FIGS. 14 and 15, respectively. These relationships reconstruct the trees of wavelet coefficient vectors, 

define vector or object wavelet trees. Qne embodiment of mis invention (as shown fa FIG 12) 

Vector or object wavelet trees are similar to the wavelet 15 uses scalar quantization of the elements of the vectors of 

trees of scalar ZTE coding except that at each node of these wavelet coefficients. After quantization, each vector will 

vector or object wavelet trees there is a vector of wavelet contain either all zeros or one or more nonzero values. A 

coefficients instead of a single value. As is done in scalar vector zerotree exists where there is a zero vector at every 

ZTE coding described above, each tree is traversed depth- node of the tree. A ycctol zerotre e root is the root of a vector 

first from the root in the low-low band through the children 20 zerotree . The coefficient vector at a vector zerotree root is 

and grandchildren. As nodes are traversed, coefficients are the zero vector and all descendants of the vector zerotree 

quantized according to the current quantization specifica- root m themselves vector zerotree roots, 

tion. This quantization can be done as a vector of scalar rQOts afe . aQt because wave]et ^ caQ fae 

quantizations by scalar quantizer 1210 or as a single vector d at 2erQtree rQOts and UQ biu are tQ 

quantization by vector quantizer U10. Various vector quan- * code thc ^efficients that have been pruned away. Therefore, 

tization techniques can be applied as disclosed by Allen tQ codi cffid6 it is desirablc to havc as many 

Gersho et al. in Vector Quantization and Signal zerotrees ^ t ization M ible . 0 ne method is to 

Compression , Kluwer Academic Publishers, (1992). cmploy a « pos t- q uantization processor" 1211 (1011) to 

Similar to the scalar ZTE coding as described above, it is perform some post-quantization processing (as illustrated in 

beneficial to reorganize the coefficient vectors of each wave- FIGS. 10 and 12) of each vector in an effort to change it to 

let tree to form a "wavelet block" as shown in FIG. 16. To the zero vector when doing so would cause only negligible 

facilitate the depth-first scanning pattern, the invention may additional distortion to the final reconstructed frame. A 

optionally reorganize the quantized coefficients of each measure of each vector can be computed and when below 

wavelet tree to form a "wavelet block" 1604. As shown in som e threshold, the vector would be set to zero. 

FIGS. 11 and 12, the reorganization is ac^mplished in the ^ scalar quantizat ion of the vector elements and 

vectors/objects wavelet tree ■reorganizer 1108 (1208)] prior to post-quantization processing to force vectors to the zero 

quantization. Since reorganization is optional, both FIGS. 11 vectors wher e appropriate, scanning and classification of the 

and 12 depict the optional nature of the reorganizer as path tree nodes is done ^ a manner simi]ar t0 that in scalar ZTE 

1106 and path 1206 which respectively bypass the reorga- codin Each Qode is assi d Qne of thfee bols . 

nizer and its associated function. root « „ valued zef0tree and „ value „ ThMB 

FIG. 16 schematically depicts a wavelet block 1604 that symbols have the same meaning as in scalar ZTE coding, 

is generated by the invention. Each wavelet block 1604 of a except that the nodes are vectors and the classification of 

frame comprises those coefficient vectors at all scales and zero or nonzero refers to the whole vector. An entropy coder 

orientations that represent the frame at the spatial location of 45 is used to code the symbols and the tables of values. The 

that block organized in ascending frequency order from tables of values are constructed as in scalar ZTE coding. To 

parent to children to grandchildren. Namely, the invention entropy code the values requires using vector entropy coding 

maps a tree 1602 extending from a vector 1600 in the instead of scalar entropy coding. 

low-low band 1606 (LL3) in the wavelet tree 1602 into a 0ne embodiment of this invention (as shown in FIG. 11) 

wavelet block 1604. Such a structure permits the quantiza- 5Q uses vec t 0 r quantization of the vectors of wavelet coeffi- 

tion factor for the case of scalar quantization, or the cieats> vector quantization (VQ) is a mature field and 

codebook, for the case of vector quantization, to adapt to various forai s of VQ desired can be used. As discussed 

each block according to where the block is spatially located above> it ^ desirable to increase the number of zerotree roots 

and thus what it represents in the frame. when doing M does not sigrjificanlly mcrease the distortion 

By using a depth-first scanning pattern, each wavelet 55 of the reconstructed frame. To achieve this result, the VQ 

block is completely scanned to quantize its coefficient can be adapted to incorporate a form of dead zone whereby 

vectors before the next block is scanned and so on. For vectors with a measure below some threshold are mapped to 

example, block 1610 is completely scanned, then block the index that reconstructs as the zero vector. 

1612, then block 1614, and so on in a raster scan pattern Vector quantization replaces the vector at each node of the 

through the frame of wavelet blocks. The ordering of blocks 60 tree with an index into a codebook of vectors that selects the 

does not have to be in a raster scan pattern, but can be any best reconstruction vector for that node. After vector 

order as desired by the application. quantization, scanning and classification of the tree nodes is 

Furthermore, this invention presents two distinct methods done in a manner similar to that in scalar ZTE coding. Each 

for performing the quantization of the vectors of wavelet node is assigned one of three symbols: "zerotree root", 

coefficients. Quantization can be performed as scalar quan- 65 "valued zerotree root", and "value". These symbols have the 

tization on each element of each vector, or it can be same meaning as in scalar ZTE coding, except that the nodes 

performed as vector quantization on each vector. At each are now indices into a codebook and zero or nonzero refers 
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to the codebook entry for each index. An entropy coder is 
used to code the symbols and the tables of indices. The 
tables of indices are constructed as in scalar ZTE coding. 

FIGS. 9 and 10 illustrate alternative embodiments of the 
present invention that use the vector wavelet transform. 
Again, since the encoders 900 and 1000 of FIGS. 9 and 10 
are similar to the encoders of FIGS. 11 and 12 respectively, 
devices 908-914 and 1008-1014 are not discussed. Since 
these devices perform the same functions as those of 
1108-1114 and 1208-1214 respectively, the descriptions for 
these devices are identical to the discussion above. 

However, unlike encoders 1100 and 1200, the encoders 
900 and 1000 employ a vector organizer 903 (1003) in 
conjunction with a vector wavelet tree generator 904 (1004), 
as discussed above with reference to FIG. 13. After the 
vector wavelet trees are generated, the encoders 900 and 
1000 behave similarly to the encoders 1100 and 1200 
respectively. 

Those skilled in the art will realize that although the 
vector or object wavelet coding incorporates the Zerotree 
Entropy (ZTE) coding method, it can also be implemented 
with other coding method such as the Embedded Zerotree 
Wavelet (EZW) algorithm. 

Although various embodiments which incorporate the 
teachings of the present invention have been shown and 
described in detail herein, those skilled in the art can readily 
devise many other varied embodiments that still incorporate 
these teachings. 

What is claimed is: 

1. Method for organizing a vector parent-child 
relationship, said method comprising the steps of: 

(a) propagating a vector parent into a higher band by 
upsampling to define an outline; 

(b) regrouping coefficients within said outline into a 
plurality of vector children; and 

(c) extracting said vector children by taking a shifted 
subsampling of said outline. 

2. Method for organizing a vector parent-child 
relationship, said method comprising the steps of: 

(a) propagating a vector parent into a higher band by 
upsampling to define an outline; 

(b) regrouping coefficients within said outline into a 
plurality of vector children; and 

(c) extracting said vector children by taking a plurality of 
polyphase components of said outline by spatially 
offsetting and then subsampling in each of two dimen- 
sions. 

3. The method of claim 1, wherein said regrouping step 
(b) regroups coefficients within said outline into four vector 
children. 

4. The method of claim 1, wherein said propagating step 
(a) propagates an object parent into a higher band by 
upsampling to define an outline, and wherein said regroup- 
ing step (b) regroups coefficients within said outline into a 
plurality of object children. 

5. Method for organizing a vector wavelet for a frame, 
said method comprising the steps of: 



,167 Bl 

14 

(a) decomposing the frame into a plurality of polyphase 
components, wherein said decomposing step decom- 
poses the frame into a plurality of polyphase compo- 
nents by spatially offsetting the frame and then sub- 

5 sampling the offset frame in each of two dimensions; 

(b) applying a transform to each of said plurality of 
polyphase components; and 

(c) forming a vector wavelet by grouping together trans- 
form coefficients from each of said transformed 

10 polyphase components. 

6. The method of claim 5, wherein said applying step (b) 
applies a wavelet transform to each of said plurality of 
polyphase components. 

7. The method of claim 5, wherein said forming step (c) 
forms an object wavelet by grouping together transform 
coefficients from each of said transformed polyphase com- 
ponents. 

8. Apparatus for organizing a vector wavelet for a frame, 
said apparatus comprising: 

a decomposing means for decomposing the frame into a 
plurality of polyphase components, wherein said 
decomposing means decomposes the frame into a plu- 
rality of polyphase components by spatially offsetting 
the frame and then subsampling the offset frame in each 
of two dimensions; 

25 

a transforming means for transforming each of said plu- 
rality of polyphase components; and 

a forming means for forming a vector wavelet by group- 
ing together transform coefficients from each of said 
30 transformed polyphase components. 

9. The apparatus of claim 8, wherein said transforming 
means is a wavelet transform. 

10. The apparatus of claim 8, wherein said forming means 
forms an object wavelet by grouping together transform 

35 coefficients from each of said transformed polyphase com- 
ponents. 

11. A computer-readable medium having stored thereon a 
plurality of instructions, the plurality of instructions includ- 
ing instructions which, when executed by a processor, cause 

4 q the processor to perform the steps comprising of: 

(a) decomposing a frame into a plurality of polyphase 
components, wherein said decomposing step decom- 
poses the frame into a plurality of polyphase compo- 
nents by spatially offsetting the frame and then sub- 

45 sampling the offset frame in each of two dimensions; 

(b) applying a transform to each of said plurality of 
polyphase components; and 

(c) forming a vector wavelet by grouping together trans- 
form coefficients from each of said transformed 

50 polyphase components. 

12. The computer-readable medium of claim 11, wherein 
said applying step (b) applies a wavelet transform to each of 
said plurality of polyphase components. 

13. The computer-readable medium of claim 11, wherein 
55 said forming step (c) forms an object wavelet by grouping 

together transform coefficients from each of said trans- 
formed polyphase components. 

* « * * * 
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